Check that PRs' `CHANGELOG.md` changes don't modify already-released content in CI #8351

ErichDonGubler · 2025-10-14T19:38:20Z

We've had recurring issues with PRs' CHANGELOG.md entries going stale, particularly when PRs are filed close to a regular release date. This PR tries to make this by creating a CI check that fails when a PR modifies the changelog and it detects changes outside the ## Unreleased section header.

This is an MVP that is not yet suitable as a required check, because there is no escape hatch for cases where we do want to change released changelog content. The intent is to enable this once some follow-up to add such an escape hatch has achieved consensus and is implemented.

ErichDonGubler · 2025-10-22T17:06:18Z

Recent violators of the check I'm adding here that I had to fix in #8407:

ErichDonGubler · 2025-10-22T17:08:19Z

Sigh, this check is running into issues with GitHub's default checkout action machinery wanting to be as shallow as possible. The "easy" fix here is not to use shallow repository checkouts. We'll want follow-up to see if there's a way to only check out what's necessary, but that's sort of a catch-22 unless we use the GitHub API (which I'd strongly prefer to avoid, so folks can run things locally offline).

ErichDonGubler · 2025-10-22T19:25:17Z

I have several PRs demonstrating that core logic works here:

Failure case(s)
- DO NOT MERGE: CHANGELOG check: example of something in a releasd section, right after the section begins · gfx-rs/wgpu@8b4659a
Success case(s)

These are present in the check's unit tests in xtask, so I'm not worried about regressions in the future. 😁

There are some questions that are still open for this, but none are blocking until we make this check required:

What escape hatch should we use when we need this, i.e., our typos database is updated, and we correct an old CHANGELOG entry, and we want the PR to land? It's pretty easy to use a label, i.e., changelog: released entry changed, to control this. I suggest we do that.

We don't need to resolve this right now, because it's only important if we make this a required check. I don't see that as necessary for a first iteration.
What fancy GitHub integration do we want? I can see several things that look interesting:
- Step summary
- Warning/error messages on end file contents of the PR

ErichDonGubler · 2025-10-22T19:56:02Z

.github/workflows/changelog.yml

@@ -0,0 +1,42 @@
+name: Check `CHANGELOG.md`


Bikeshedding for the file's name welcome!

ErichDonGubler · 2025-10-22T19:56:06Z

.github/workflows/changelog.yml

+  changelog:
+    timeout-minutes: 1
+
+    name: Check `CHANGELOG` for errors


Bikeshedding for the job's name welcome!

xtask/src/changelog.rs

ErichDonGubler · 2025-11-05T16:54:02Z

From today's maintainers' meeting notes:

#8351 Check that PRs' CHANGELOG.md changes don't modify already-released content in CI (By Erich Gubler; Unassigned)

EG: This is coming, and it won’t immediately annoy people in CI, but the intent is to add another required check.

Consensus: All: Let’s land, as a required check.

.github/workflows/changelog.yml

xtask/src/changelog.rs

cwfitzgerald · 2025-11-05T19:54:02Z

xtask/src/changelog.rs

+
+#[cfg(test)]
+mod test_hunks_in_a_released_section {
+    #[collapse_debuginfo(yes)]


What dafaq is this?

It's a dissting!

When macros are used outside the crate defining them, this is the default. Within a crate, however, compiler errors spans, by default, will point to expanded code, not use sites. I felt it was nicer to do this from a testing standpoint, but it's not absolutely necessary, as the tests are working.

~~I just noticed we don't seem to have coverage for xtask unit tests in CI… 🤔~~

EDIT: That's not right, sigh.

cwfitzgerald · 2025-11-05T19:55:12Z

xtask/src/changelog.rs

+use pico_args::Arguments;
+use xshell::Shell;
+
+pub(crate) fn check_changelog(shell: Shell, mut args: Arguments) -> anyhow::Result<()> {


Do we need a check to make sure your working state isn't dirty? Is this destructive to the working copy?

We might, but I don't know if it's worth handling the complexity. Could you tell me what scenario you're trying to prevent?

Do changes

"I wonder if I did the changelog right"

cargo xtask changelog

Where changes?

enters rage

🤔 I think that case should be handled just fine by the user not specifying the <to_commit> positional argument, in which case the CHANGELOG.md in the filesystem is used. Does that sound like it works?

To more definitively answer your question, there's no mutation of the working copy happening here.

xtask/src/changelog.rs

ErichDonGubler force-pushed the erichdongubler-push-glitchy-unique-tambourine branch 7 times, most recently from 58ba751 to 8b3d6ff Compare October 14, 2025 20:41

ErichDonGubler force-pushed the erichdongubler-push-glitchy-unique-tambourine branch 5 times, most recently from 556a765 to e686fa7 Compare October 22, 2025 19:23

ErichDonGubler force-pushed the erichdongubler-push-glitchy-unique-tambourine branch 6 times, most recently from c13a16a to 8f102f2 Compare October 22, 2025 19:53

ErichDonGubler commented Oct 22, 2025

View reviewed changes

ErichDonGubler marked this pull request as ready for review October 22, 2025 20:06

ErichDonGubler force-pushed the erichdongubler-push-glitchy-unique-tambourine branch 3 times, most recently from 06acbe0 to 7ea2757 Compare October 22, 2025 21:48

ErichDonGubler commented Oct 22, 2025

View reviewed changes

xtask/src/changelog.rs Outdated Show resolved Hide resolved

cwfitzgerald self-requested a review November 5, 2025 16:20

cwfitzgerald self-assigned this Nov 5, 2025

ErichDonGubler force-pushed the erichdongubler-push-glitchy-unique-tambourine branch from 7ea2757 to 8d83828 Compare November 5, 2025 19:34

cwfitzgerald reviewed Nov 5, 2025

View reviewed changes

.github/workflows/changelog.yml Outdated Show resolved Hide resolved

ErichDonGubler force-pushed the erichdongubler-push-glitchy-unique-tambourine branch from 8d83828 to 8554263 Compare November 5, 2025 19:50

cwfitzgerald reviewed Nov 5, 2025

View reviewed changes

ErichDonGubler force-pushed the erichdongubler-push-glitchy-unique-tambourine branch 4 times, most recently from dfc1280 to dc8340a Compare November 5, 2025 20:36

cwfitzgerald reviewed Nov 5, 2025

View reviewed changes

xtask/src/changelog.rs Outdated Show resolved Hide resolved

ErichDonGubler added 2 commits November 6, 2025 14:37

chore(xtask): add changelog check

4752005

test(ci): add changelog job that runs cargo xtask changelog

c37e504

ErichDonGubler force-pushed the erichdongubler-push-glitchy-unique-tambourine branch from dc8340a to c37e504 Compare November 6, 2025 19:42

Check that PRs' CHANGELOG.md changes don't modify already-released content in CI #8351

Are you sure you want to change the base?

Check that PRs' CHANGELOG.md changes don't modify already-released content in CI #8351

Conversation

ErichDonGubler commented Oct 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ErichDonGubler commented Oct 22, 2025

Uh oh!

ErichDonGubler commented Oct 22, 2025

Uh oh!

ErichDonGubler commented Oct 22, 2025

Uh oh!

ErichDonGubler Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ErichDonGubler Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ErichDonGubler commented Nov 5, 2025

Uh oh!

Uh oh!

Uh oh!

cwfitzgerald Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

ErichDonGubler Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment was marked as outdated.

Uh oh!

cwfitzgerald Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

ErichDonGubler Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

cwfitzgerald Nov 5, 2025

Choose a reason for hiding this comment

Uh oh!

ErichDonGubler Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Check that PRs' `CHANGELOG.md` changes don't modify already-released content in CI #8351

Check that PRs' `CHANGELOG.md` changes don't modify already-released content in CI #8351

ErichDonGubler commented Oct 14, 2025 •

edited

Loading

ErichDonGubler Oct 22, 2025 •

edited

Loading

ErichDonGubler Oct 22, 2025 •

edited

Loading

ErichDonGubler Nov 5, 2025 •

edited

Loading